feat: Multi-turn conversation state management with type safety #124

mattapperson · 2025-12-22T07:09:35Z

Summary

Add multi-turn conversation state management with StateAccessor pattern for pluggable persistence
Add approval workflow for tool execution (requireApproval, approveToolCalls, rejectToolCalls)
Improve type safety by replacing type casts with type guards
Refactor executeToolsIfNeeded from ~400 lines to ~110 lines by extracting 14 focused helper methods

Key Changes

New State Management Features

StateAccessor interface for pluggable state persistence (in-memory, database, etc.)
ConversationState type for tracking conversation status, messages, pending approvals
createInitialState(), updateState() utilities for state management
Support for interruption handling and resumption

Approval Workflow

Tools can specify requireApproval: true to pause execution
Call-level requireApproval function for dynamic approval logic
approveToolCalls and rejectToolCalls arrays to resume with decisions
getPendingToolCalls() and requiresApproval() methods on ModelResult

Type Safety Improvements

Type guards (isValidUnsentToolResult, isValidParsedToolCall) instead of type casts
Made resolveAsyncFunctions generic to eliminate double-casting
isEventStream type guard handles both streaming and non-streaming responses

Examples

Basic State Management

import { callModel, type ConversationState, type StateAccessor } from '@openrouter/sdk';

// In-memory state storage (use database in production)
const conversations = new Map<string, ConversationState>();

const stateAccessor: StateAccessor = {
  load: async () => conversations.get('conv-123') ?? null,
  save: async (state) => { conversations.set('conv-123', state); },
};

const result = client.callModel({
  model: 'anthropic/claude-sonnet-4',
  input: [{ type: 'text', text: 'Hello!' }],
  state: stateAccessor,
});

const text = await result.getText();
const state = await result.getState(); // Access conversation state

Tool with Approval Required

import { tool } from '@openrouter/sdk';

const deleteFileTool = tool({
  name: 'delete_file',
  description: 'Delete a file from the filesystem',
  inputSchema: z.object({
    path: z.string().describe('File path to delete'),
  }),
  outputSchema: z.object({ success: z.boolean() }),
  requireApproval: true, // Pause for human approval
  execute: async ({ path }) => {
    await fs.unlink(path);
    return { success: true };
  },
});

const result = client.callModel({
  model: 'anthropic/claude-sonnet-4',
  input: [{ type: 'text', text: 'Delete the temp.txt file' }],
  tools: [deleteFileTool],
  state: stateAccessor,
});

// Check if approval is needed
if (await result.requiresApproval()) {
  const pending = await result.getPendingToolCalls();
  console.log('Pending approval:', pending);
  // Show to user, get decision...
}

Resume with Approval Decision

// After user approves/rejects...
const result = client.callModel({
  model: 'anthropic/claude-sonnet-4',
  input: [], // Empty - resuming from state
  tools: [deleteFileTool],
  state: stateAccessor,
  approveToolCalls: ['call_abc123'],  // Approve specific tool calls
  rejectToolCalls: ['call_def456'],   // Reject others
});

const text = await result.getText(); // Continues execution

Dynamic Approval Logic

const result = client.callModel({
  model: 'anthropic/claude-sonnet-4',
  input: [{ type: 'text', text: 'Process these files' }],
  tools: [readFileTool, writeFileTool, deleteFileTool],
  state: stateAccessor,
  // Dynamic approval based on tool and context
  requireApproval: (toolCall, context) => {
    // Always approve read operations
    if (toolCall.name === 'read_file') return false;
    // Require approval for destructive operations
    if (toolCall.name === 'delete_file') return true;
    // Require approval after 3 turns
    return context.numberOfTurns > 3;
  },
});

Stop Conditions

import { stepCountIs, hasToolCall, maxCost } from '@openrouter/sdk';

const result = client.callModel({
  model: 'anthropic/claude-sonnet-4',
  input: [{ type: 'text', text: 'Research this topic thoroughly' }],
  tools: [searchTool, summarizeTool],
  // Stop when any condition is met
  stopWhen: [
    stepCountIs(10),           // Max 10 tool execution rounds
    hasToolCall('summarize'),  // Stop when summarize is called
    maxCost(0.50),             // Budget limit
  ],
});

Test plan

All 320 existing tests pass
New unit tests for conversation state utilities (21 tests)
New E2E tests for state management integration (5 tests)
Build passes with no TypeScript errors
Lint passes with no warnings

…mprovements - Add StateAccessor interface for pluggable state persistence - Add ConversationState type with status tracking (complete, interrupted, awaiting_approval, in_progress) - Add approval workflow support with requireApproval tool option and partitionToolCalls helper - Add type guards (isValidUnsentToolResult, isValidParsedToolCall) to replace unsafe type assertions - Make resolveAsyncFunctions generic over TTools to eliminate double-casting - Fix isEventStream to check constructor name on prototype chain - Handle both streaming and non-streaming API responses gracefully - Add conversation-state.ts with helper functions for state management - Add unit tests for conversation state utilities (21 tests) - Add e2e tests for state management integration (5 tests)

Extract 14 helper methods from the monolithic executeToolsIfNeeded method to improve maintainability and reduce complexity: - getInitialResponse(): Get initial response from stream or cache - saveResponseToState(): Save response output to state - markStateComplete(): Mark conversation complete - saveToolResultsToState(): Save tool results to state - checkForInterruption(): Handle interruption signals - shouldStopExecution(): Evaluate stop conditions - hasExecutableToolCalls(): Check for executable tools - executeAutoApproveTools(): Execute non-approval tools - handleApprovalCheck(): Deduplicated approval workflow - executeToolRound(): Execute one round of tools - resolveAsyncFunctionsForTurn(): Resolve async params per turn - applyNextTurnParams(): Apply nextTurnParams - makeFollowupRequest(): Build and send follow-up requests - validateFinalResponse(): Validate final response The main method is reduced from ~400 lines to ~110 lines, acting as a clean coordinator that delegates to these focused helpers. Also replaces 'name as string' casts with String() for safer conversion.

Update the call-level requireApproval function signature to: - Accept TurnContext as a second parameter - Support async functions (returning Promise<boolean>) This enables approval decisions based on conversation state: - Number of turns completed - Dynamic approval logic based on context Updated files: - src/lib/async-params.ts - Update type definition - src/lib/model-result.ts - Update type and field - src/lib/conversation-state.ts - Make toolRequiresApproval and partitionToolCalls async, add context parameter - tests/unit/conversation-state.test.ts - Update tests, add async test - tests/e2e/call-model-state.test.ts - Update test signature

The tool-level requireApproval now supports both boolean values and async functions that receive the tool's input params and turn context, matching the call-level requireApproval signature. This enables dynamic approval decisions based on: - Tool arguments (e.g., require approval only for dangerous parameter values) - Turn context (e.g., require approval after a certain number of turns) Changes: - Add ToolApprovalCheck type for function-based approval - Update BaseToolFunction and all tool config types - Update toolRequiresApproval to handle function-based approval - Add unit tests for function-based tool-level approval - Export ToolApprovalCheck type from index

Add type-level and runtime enforcement that approval-related parameters require a state accessor: Type-level enforcement: - CallModelInput now uses conditional types so `approveToolCalls` and `rejectToolCalls` are typed as `never` when `state` is not provided - Added helper types: ToolHasApproval, HasApprovalTools Runtime enforcement: - Throws error in ModelResult constructor if approveToolCalls/rejectToolCalls provided without state accessor - Throws error in handleApprovalCheck if tools require approval but no state accessor is configured New exports: - Type guards: toolHasApprovalConfigured, hasApprovalRequiredTools - Helper types: ToolHasApproval, HasApprovalTools, CallModelInputWithApprovalTools

…hState

- Added missing default stop condition in shouldStopExecution() that was documented but not implemented, preventing infinite tool execution loops - Changed test model from llama-3.1-8b-instruct to claude-sonnet-4.5 to fix timeout issues in chat-style tools test

Resolve conflicts: - tool-executor.ts: Keep zod v4 type compatibility fix (as any cast) - call-model.test.ts: Use anthropic/claude-haiku-4.5 model

- Extract duplicate async resolution to resolveRequestForContext() helper - Add saveStateSafely() wrapper with error handling for state persistence - Create clearOptionalStateProperties() for consistent state cleanup - Extract DEFAULT_MAX_STEPS constant (was magic number 5) - Fix numberOfTurns to be consistently 1-indexed - Remove unnecessary type cast in call-model.ts - Add comprehensive JSDoc to all helper methods - Fix const to let for reassigned baseRequest variable

Resolves merge conflicts by: - Combining conversation state exports with tool event broadcaster export - Keeping multi-turn state management while integrating real-time preliminary result streaming via broadcaster callback - Using correct z4.toJSONSchema with validation from main

mattapperson added 7 commits December 22, 2025 01:51

refactor: rename CallModelInputWithApprovalTools to CallModelInputWit…

6729753

…hState

subtleGradient self-requested a review December 25, 2025 16:18

mattapperson added 2 commits January 6, 2026 10:28

Merge origin/main into multi-turn

1bd83e0

Resolve conflicts: - tool-executor.ts: Keep zod v4 type compatibility fix (as any cast) - call-model.test.ts: Use anthropic/claude-haiku-4.5 model

mattapperson requested review from alexanderatallah and louisgv January 6, 2026 16:57

mattapperson requested a review from robert-j-y January 7, 2026 20:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Multi-turn conversation state management with type safety #124

feat: Multi-turn conversation state management with type safety #124

Uh oh!

mattapperson commented Dec 22, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: Multi-turn conversation state management with type safety #124

Are you sure you want to change the base?

feat: Multi-turn conversation state management with type safety #124

Uh oh!

Conversation

mattapperson commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Changes

New State Management Features

Approval Workflow

Type Safety Improvements

Examples

Basic State Management

Tool with Approval Required

Resume with Approval Decision

Dynamic Approval Logic

Stop Conditions

Test plan

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mattapperson commented Dec 22, 2025 •

edited

Loading